IMPEDANCE SENSING SCREEN POINTING DEVICE 
Reference to Related Patents 

This Application is related to the subject matter described in the 
following U.S. patents: U.S. Patent No. 5,578,813, filed March 2, 1995, issued 
5 November 26, 1996, and entitled FREEHAND IMAGE SCANNING DEVICE 
WHICH COMPENSATES FOR NON-LINEAR MOVEMENT; U.S. Patent No. 
5,644,139, filed August 14, 1996, issued July 1, 1997, and entitled 
NAVIGATION TECHNIQUE FOR DETECTING MOVEMENT OF 
NAVIGATION SENSORS RELATIVE TO AN OBJECT; and U.S. Patent No. 

1 0 5,786,804, filed October 6, 1 995, issued July 28, 1 998, and entitled METHOD 
AND SYSTEM FOR TRACKING ATTITUDE. These three patents describe 
techniques of tracking position movement and computer pointing devices. 
Those techniques are a component in a preferred embodiment described below. 
Accordingly, U.S. Pat. Nos. 5,578,813, 5,644,139, and 5,786,804 are hereby 

1 5 incorporated herein by reference. This application is also related to the subject 
matter described in U.S. Patent No. 6,057,540, filed April 30, 1998, issued May 
2, 2000, and entitled MOUSELESS OPTICAL AND POSITION 
TRANSLATION TYPE SCREEN POINTER CONTROL FOR A COMPUTER 
SYSTEM; U.S. Patent No. 6,151,015, filed April 27, 1998, issued November 21, 

20 2000, and entitled PEN LIKE COMPUTER POINTING DEVICE; and U.S. 

patent application Ser. No. 09/052,046, filed March 30, 1998, entitled SEEING 
EYE MOUSE FOR A COMPUTER SYSTEM. These two related patents and 
patent application describe screen pointing devices based on the techniques 
described in U.S. Pat. Nos. 5,578,813, 5,644,139, and 5,786,804. Therefore, 

25 U.S. Patent Nos. 6,057,540 and 6,151,015, and U.S. patent application Ser. No. 
09/052,046, filed March 30, 1998, entitled SEEING EYE MOUSE FOR A 
COMPUTER SYSTEM, are hereby incorporated herein by reference. 



The Field of the Invention 

30 This invention relates generally to devices for controlling a cursor on a 

display screen, also known as pointing devices. This invention relates more 
particularly to an impedance sensing screen pointing device. 
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Background of the Invention 

The use of a hand operated pointing device for use with a computer and 
its display has become almost universal. By far the most popular of the various 
devices is the conventional (mechanical) mouse, used in conjunction with a 
5 cooperating mouse pad. Centrally located within the bottom surface of the 
mouse is a hole through which a portion of the underside of a rubber-surfaced 
steel ball extends. The mouse pad is typically a closed cell foam rubber pad 
covered with a suitable fabric. Low friction pads on the bottom surface of the 
mouse slide easily over the fabric, but the rubber ball does not skid. Rather, the 

1 0 rubber ball rolls over the fabric as the mouse is moved. Interior to the mouse are 
rollers, or wheels, that contact the ball at its equator and convert its rotation into 
electrical signals representing orthogonal components of mouse motion. These 
electrical signals are coupled to a computer, where software responds to the 
signals to change by a AX and a AY the displayed position of a pointer (cursor) 

15 in accordance with movement of the mouse. The user moves the mouse as 

necessary to get the displayed pointer to a desired location or position. Once the 
pointer on the screen points at an object or location of interest, a button on the 
mouse is activated with the fingers of the hand holding the mouse. The 
activation serves as an instruction to take some action, the nature of which is 

20 defined by software in the computer. 

A "track ball" is another example of a mechanical type of pointing 
device. A track ball is essentially an upside-down mouse. In a track ball, rather 
than sliding the device itself over a surface to produce pointer movement as in a 
mouse, a user directly contacts the mechanical ball with the user's finger, and 

25 causes the ball to rotate. As with a mouse, the movement of the mechanical ball 
in a track ball generates a corresponding movement of the displayed pointer. 

Conventional mechanical pointing devices, such as those described 
above, have a number of shortcomings. Among the shortcomings are 
deterioration of the mouse ball or damage to its surface, deterioration or damage 

30 to the surface of the mouse pad, and degradation of the ease of rotation of the 
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contact rollers because of accumulation of dirt or lint, and because of wear. 
These problems can contribute to erratic operation or total failure of the mouse. 

In addition to mechanical types of pointing devices like a conventional 
mouse, optical pointing devices have also been developed, such as those 
5 described in the incorporated patents and patent application. In one form of an 
optical pointing device, rather than using a moving mechanical element like a 
ball in a conventional mouse, relative movement between an imaging surface, 
such as a finger or a desktop, and photo detectors within the optical pointing 
device, is optically sensed and converted into movement information. Optical 
1 0 pointing devices typically use a light source, such as an LED, to illuminate an 
imaging surface. Optical pointing devices also typically use lenses for focusing 
images, which requires a certain amount of depth or thickness of the pointing 
device. 

It would be desirable to provide a screen pointing device that is non- 
1 5 mechanical in operation, that is more compact than conventional pointing 

devices, and that does not use a light source for illuminating an imaging surface, 
thereby allowing lower power to be used. 

Summary of the Invention 

One form of the present invention provides an apparatus for controlling 
the position of a screen pointer for an electronic device having a display screen. 
The apparatus includes a plurality of sensing elements against which a portion of 
the tip of a human digit may be placed. A controller coupled to each of the 
sensing elements senses an electrical property at each of the sensing elements. 
The controller is configured to generate values representing the portion of the tip 
of the digit placed against the sensing elements based on the sensed electrical 
property at each of the sensing elements. The controller is configured to 
generate movement data based on the values. The movement data is indicative 
of motion of the tip of the digit across the sensing elements. 
20 Another form of the present invention provides a method of controlling 

the position of a screen pointer for an electronic device having a screen display. 
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A portion of an appendage of the human hand is placed against a plurality of 
sensing elements. An impedance is sensed at each of the sensing elements. 
Digital values representing the sensed impedance at each of the sensing elements 
are generated. The digital values represent digital images of the portion of the 
5 appendage placed against the sensing elements. At least one version of a first 
one of the digital images is correlated with at least one version of a second one 
of the digital images to generate motion data indicative of motion across the 
sensing elements by the appendage. The position of the screen pointer is 
adjusted in accordance with the motion data. 

10 Brief Description of the Drawings 

Figure 1 is a pictographic side view of the main components of a prior art 
optical, motion translation type screen pointer device. 

Figure 2 is a perspective view of one embodiment of a screen pointing 
device according to the present invention. 
15 Figure 3 is an electrical block diagram illustrating major components of 

one embodiment of a screen pointing device according to the present invention. 

Description of the Preferred Embodiments 

In the following detailed description of the preferred embodiments, 
20 reference is made to the accompanying drawings, which form a part hereof, and 
in which is shown by way of illustration specific embodiments in which the 
invention may be practiced. It is to be understood that other embodiments may 
be utilized and structural or logical changes may be made without departing 
from the scope of the present invention. The following detailed description, 
25 therefore, is not to be taken in a limiting sense, and the scope of the present 
invention is defined by the appended claims. 

Figure 1 shows a simplified representation of a side view of a prior art 
motion detection arrangement 1 suitable for tracking the movement of a human 
finger 7 pressed against a surface 5 of a transparent stud 3. A motion detection 
30 arrangement like that shown in Figure 1 is described in detail in the above- 
incorporated U.S. Patent No. 6,057,540 (the '540 patent). The operation of 
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motion detection arrangement 1 is also summarized below. 

When the tip 6 of finger 7 is pressed against surface 5 5 the ridges of skin 
and any other micro texture features are visible in the plane of surface 5, just as 
if they were a part of surface 5. Lens 8 focuses light from those features onto an 
5 array of photo detectors, which is part of movement sensor 9. Movement sensor 
9 automatically acquires and tracks any suitable image. When tracking an 
image, movement sensor 9 produces incremental (X, Y) signals. 

Lifting fingertip 6 away from surface 5 produces a loss of tracking. This 
condition is detected within motion detector 9, and in one embodiment, the 

10 production of incremental (X, Y) signals ceases. This has the effect of leaving 
the position of the screen pointer unchanged at whatever location it currently 
occupies, and is exactly the same as when a user of a mouse removes his hand 
from the mouse. When fingertip 6 is subsequently replaced on surface 5, motion 
detector 9 appreciates that an image has been acquired, and, in one embodiment, 

15 treats that acquisition as though a reset has been performed. That is, until there 
has been new motion subsequent to the new acquisition, the incremental 
coordinates (X, Y) will have the value (0, 0). This leaves the existing position of 
the screen pointer undisturbed until such time as it is deliberately moved by the 
motion of fingertip 6, and corresponds exactly to a mouse user's placement of 

20 his hand back on the mouse without moving it. 

An LED 2, which is an IR LED in one embodiment, emits light that is 
projected by lens 4 onto a region 5 that is part of a work surface 6 to be imaged 
for navigation. In one embodiment, motion sensor 9 is an integrated circuit (IC) 
having an array of photo detectors, memory, and arithmetic circuits arranged to 

25 implement image correlation and tracking functions described herein and in the 
incorporated patents. An image of the illuminated region 6 is projected through 
an optical window (which may be transparent stud 3 itself) to a package (not 
shown) of integrated circuit 9 and onto the array of photo detectors. Lens 8 aids 
in the projection of the image onto the photo detectors. 

30 One preferred optical navigation technique used by motion detection 

arrangement 1 involves optically detecting motion by directly imaging as an 
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array of pixels the various particular optical features visible at surface 5, much as 
human vision is believed to do. IR light reflected from a textured work surface 
pressed against surface 5 is focused onto a suitable array (e.g., 16x16 or 24x24) 
of photo detectors. The responses of the individual photo detectors are digitized 
5 to a suitable resolution (e.g., six or eight bits) and stored as a frame into 

corresponding locations within an array of memory. In one embodiment, each 
pixel in a frame corresponds to one of the photo detectors. 

The overall size of the array of photo detectors is preferably large enough 
to receive an image having several features (e.g., ridges in the whorls of skin). 

10 In this way, images of such spatial features produce translated patterns of pixel 
information as fingertip 6 moves. The number of photo detectors in the array 
and the frame rate at which their contents are digitized and captured cooperate to 
influence how fast fingertip 6 can be moved over surface 5 and still be tracked. 
Tracking is accomplished by comparing a newly captured sample frame with a 

1 5 previously captured reference frame to ascertain the direction and amount of 
movement. 

Figure 2 is a perspective view of one embodiment of a screen pointing 
device according to the present invention. As can be seen in Figure 2, screen 
pointing device 1 00 is more compact than the arrangement 1 shown in Figure 1 . 

20 Screen pointing device 100 is not shown actual size in Figure 2, but rather it is 
substantially enlarged to better illustrate components of the device. Screen 
pointing device 100 includes an array of sensing elements 102, guard layer 104, 
substrate 106, and controller 108. In one embodiment, screen pointing device 
100 is integrated into a single chip. Sensing elements 102 are formed on 

25 substrate 106. In one embodiment, sensing elements 102 are configured in a 
16x16 array, with the array being about 1.5 millimeters by 1.5 millimeters. 
Other configurations may be used in alternative embodiments, including a 24x24 
or other size array. In one embodiment, the pitch of sensing elements 102 ranges 
between 25 and 250 microns. In another form of the invention, the pitch of 

30 sensing elements ranges between 10 and 1000 microns. 
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In one form of the invention, each sensing element 102 includes a 
conductive layer 102B formed on substrate 106, and a protective layer 102 A 
formed over the conductive layer 102B. In an alternative embodiment, 
protective layer 1 02 A is a single protective layer sheet covering all conductive 
5 layers 102B. Conductive layer 102B is a metal, and protective layer 102 A is an 
insulator, in one embodiment. Guard layer 104 is formed on substrate 106 
around the array of sensing elements 102. Controller 108 is coupled to each 
sensing element 102, and to guard layer 104. In one embodiment, conductive 
traces are provided in substrate 1 06 to couple controller 1 08 to sensing elements 

10 1 02 and guard layer 1 04. 

Like the prior art motion detection arrangement shown in Figure 1 , 
screen pointing device 100 also essentially acquires "images" of a fingertip 6, 
and uses the images to detect movement of fingertip 6 across sensing elements 
102. However, rather than using optical techniques to illuminate fingertip 6 and 

1 5 generating images based on reflected light, screen pointing device 100 directly 
generates images or digital representations of fingertip 6 based on sensed 
electrical properties at sensing elements 102, as described in further detail below. 
Controller 108 produces incremental (X, Y) signals indicative of movement of 
fingertip 6 across sensing elements 102. The incremental (X, Y) signals may be 

20 used by a computer or other electronic device to move a screen pointer on a 
display screen. 

In one embodiment, after images have been captured, controller 1 08 uses 
exactly or substantially the same technique as the imaging and navigation 
arrangement described in the incorporated Patents to correlate the images and 
25 determine movement of fingertip 6 across sensing elements 102. In addition to 
being described in the incorporated patents, one embodiment of a technique for 
correlating images and determining movement information is also summarized 
below. 

To capture an image of fingertip 6 in one form of the present invention, 
30 controller 108 measures the capacitance to ground of each conductive layer 
102B when fingertip 6 contacts sensing elements 102. The capacitance 
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measurement varies for each sensing element 102 based upon the characteristics 
of the portion of fingertip 6 that contacts the sensing element 102. The surface 
of a finger tip 6 includes a plurality of humps and valleys that form whorls. 
When a valley is positioned over a sensing element 102, a different capacitance 
5 measurement is obtained than when a hump is positioned over a sensing element 
102. The capacitance will be greater when a hump of a whorl is positioned over 
a sensing element 1 02 than when a valley of a whorl is positioned over the 
sensing element 102. 

Those of ordinary skill in the art are aware of various techniques for 

10 measuring capacitance. In one form of the invention, capacitance to ground of a 
sensing element 102 is measured by controller 108 by driving fingertip 6 with an 
alternating current (AC) signal, and sensing the charge accumulated on each 
sensing element 102. In one embodiment, guard layer 104 is coupled to the AC 
signal, and the AC signal is passed on to fingertip 6 when fingertip 6 contacts 

15 guard layer 104. In one embodiment, in addition to, or as an alternative to, 
sensing capacitance at sensing elements 1 02, another electrical property or 
impedance is sensed and used to generate values representing portions of 
fingertip 6. 

Figure 3 shows an electrical block diagram illustrating major components 
20 of controller 108. Controller 108 includes electronic shutter 150 having a first 
plurality of switches 151 A- 151 C (collectively referred to as switches 151) and a 
second plurality of switches 153A-153C (collectively referred to as switches 
153). Switches 151 are operated independently of switches 153. Controller 108 
also includes a plurality of sense capacitors 1 54A-154C (collectively referred to 
25 as sense capacitors 154), multiplexer 156, analog to digital (A/D) converter 158, 
correlator 160, AC signal generator 162, and shutter control and automatic gain 
controller (AGC) 164. 

In operation, according to one embodiment, AC signal generator 1 62 
drives guard layer 104, and correspondingly fingertip 6, with an AC signal. 
30 Fingertip 6 essentially includes a plurality of variable capacitive elements. The 
portion of fingertip 6 above each sensing element 102 acts as a variable 
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capacitive element. In Figure 3, fingertip 6 is represented by a plurality of 
variable capacitive elements 148. To simplify the illustration and explanation, 
only three variable capacitive elements 148 and three conductive layers 102B are 
shown in Figure 3, although the actual number will depend upon the number of 
5 sensing elements 102 used. When shutter switches 151 are open and shutter 

switches 153 are closed, sense capacitors 154 and the variable capacitors created 
between fingertip 6 and conductive layers 102B form a capacitive potential 
divider. The charge that accumulates on sense capacitors 1 54 creates a voltage 
that is related to the size of the capacitances formed between the fingertip 6 and 

10 each conductive layer 102B. When shutter switches 153 are opened, no further 
charge accumulates or is lost from sense capacitors 1 54. After shutter switches 
153 are opened, conductive layers 102B are coupled to ground by closing 
switches 151 to discharge conductive layers 102B. Multiplexer 156 connects 
each sense capacitor 154 in turn to A/D converter 158, to convert the voltage on 

15 each sense capacitor 154 into a digital value. Sense capacitors 154 are then 

coupled to ground by closing switches 151 and 153 to discharge sense capacitors 
154. Switches 151 are then opened and switches 1 53 are closed so that the 
charging process can be repeated. 

Based on the level of capacitance of each sensing element 102, which is 

20 temporarily stored in sense capacitors 154, A/D converter 158 generates a digital 
value of a suitable resolution (e.g., one to eight bits) indicative of the level of 
capacitance. The digital values for the 16x16 array of sensing elements 102 
represent a digital image or digital representation of the portion of fingertip 6 
positioned over the array of sensing elements 102. The digital values for the 

25 16x16 array of sensing elements 102 are stored as a frame into corresponding 
locations within an array of memory within correlator 160. In one embodiment, 
each pixel in a frame corresponds to one of the sensing elements 102. 

The overall size of the array of sensing elements 102 is preferably large 
enough to receive an image having several features (e.g., ridges in the whorls of 

30 skin). In this way, images of such spatial features produce translated patterns of 
pixel information as fingertip 6 moves. The number of sensing elements 102 in 
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the array and the frame rate at which their contents are digitized and captured 
cooperate to influence how fast fingertip 6 can be moved across sensing 
elements 102 and still be tracked. Tracking is accomplished by correlator 160 
by comparing a newly captured sample frame with a previously captured 
5 reference frame to ascertain the direction and amount of movement. 

In one embodiment, the entire content of one of the frames is shifted by 
correlator 160 by a distance of one pixel successively in each of the eight 
directions allowed by a one pixel offset trial shift (one over, one over and one 
down, one down, one up, one up and one over, one over in the other direction, 

10 etc.). That adds up to eight trials. Also, since there might not have been any 

motion, a ninth trial "null shift" is also used. After each trial shift, those portions 
of the frames that overlap each other are subtracted by correlator 1 60 on a pixel 
by pixel basis, and the resulting differences are preferably squared and then 
summed to form a measure of similarity (correlation) within that region of 

15 overlap. Larger trial shifts are possible, of course (e.g., two over and one down), 
but at some point the attendant complexity ruins the advantage, and it is 
preferable to simply have a sufficiently high frame rate with small trial shifts. 
The trial shift with the least difference (greatest correlation) can be taken as an 
indication of the motion between the two frames. That is, it provides raw 

20 movement information that may be scaled and or accumulated to provide display 
pointer movement information (AX and AY) of a convenient granularity and at a 
suitable rate of information exchange. In one embodiment, analog to digital 
conversion is not performed, and correlation is performed on analog data. 

Correlator 160 automatically detects when fingertip 6 has been removed 

25 from sensing elements 102, by sensing that all or a majority of the pixels in the 
image have become essentially uniform. When fingertip 6 is not in contact with 
the array of sensing elements 102, the capacitance to ground of each sensing 
element 102 will be close to the same value, which results in near uniform pixel 
values. 

30 In addition to providing digital images to correlator 160, A/D converter 

158 also outputs digital images to shutter control and automatic gain controller 
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164. Shutter control and automatic gain controller 164 helps to ensure that 
successive images have a similar exposure, and helps to prevent the digital 
values from becoming saturated to one value. Controller 164 checks the values 
of digital images and determines whether there are too many minimum values or 
5 too many maximum values. If there are too many minimum values, controller 
164 either increases the output of AC signal generator 162, or increases the 
charge accumulation time of electronic shutter 150, or increases both. If there 
are too many maximum values, controller 164 either decreases the output of AC 
signal generator 162, or decreases the charge accumulation time of electronic 

10 shutter 150, or decreases both. 

In operation, images should be acquired at a rate sufficient that 
successive images differ in distance by no more that perhaps a quarter of the 
width of the array, or 4 pixels for a 16x16 array of sensing elements 102. 
Experiments show that a finger speed of 50 mm/sec is not unreasonable, which 

1 5 corresponds to a speed at the array of 800 pixels per second. To meet a 

requirement of not moving more than four pixels per cycle, a measurement rate 
of 200 samples per second is needed. This rate is quite practical, and it may be 
desirable to operate at several times this rate. 

In one form of the invention, rather than driving guard layer 104 and 

20 fingertip 6 with an AC signal from AC signal generator 162, fingertip 6 is driven 
by signals picked up from the surrounding air. The body acts as antenna and 
receives signals from the air, such as from nearby transmission lines. In this 
embodiment, guard layer 104 need not be present, or may be present but is not 
driven by AC signal generator 162. In one form of the invention, if fingertip 6 is 

25 driven by signals from the surrounding air, shutter control and automatic gain 
controller 164 operates synchronously with the signal, and controls electronic 
shutter 150 to provide an appropriate charge accumulation time for the signal. 

It will be understood by a person of ordinary skill in the art that functions 
performed by controller 1 08 may be implemented in hardware, software, 

30 firmware, or any combination thereof. The implementation may be via a 

microprocessor, programmable logic device, or state machine. Components of 
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the present invention may reside in software on one or more computer-readable 
mediums. The term computer-readable medium as used herein is defined to 
include any kind of memory, volatile or non- volatile, such as floppy disks, hard 
disks, CD-ROMs, flash memory, read-only memory (ROM), and random access 
5 memory. 

Embodiments of the present invention provide numerous benefits, 
including the use of low power, since no light source is needed to generate 
images, and a compact size, since optical elements such as lenses, which 
contribute to the depth of an optical screen pointing device, are not needed to 

10 focus generated images. Because of the lower power requirements and compact 
nature, one form of the present invention is ideally suited for portable electronic 
devices such as cellular telephones, personal digital assistants (PDAs), portable 
music players (e.g., MP3 players), pagers, portable game devices, and other 
portable devices. In addition, embodiments of the present invention provide 

1 5 additional functionality that is not present in current fingerprint sensor 

technologies. For example, some fingerprint sensors use capacitive sensing, but 
they do not track movement of a fingertip and correlate successive images to 
generate movement information. In addition, embodiments of the sensor of the 
present invention are over ten times smaller in size and number of pixels than 

20 many existing fingerprint sensors (e.g., 1.5 millimeters with 16x16 pixels 
versus 20mm and 500 x 500 pixels). 

Although specific embodiments have been illustrated and described 
herein for purposes of description of the preferred embodiment, it will be 
appreciated by those of ordinary skill in the art that a wide variety of alternate 

25 and/or equivalent implementations may be substituted for the specific 

embodiments shown and described without departing from the scope of the 
present invention. Those with skill in the chemical, mechanical, electro- 
mechanical, electrical, and computer arts will readily appreciate that the present 
invention may be implemented in a very wide variety of embodiments. This 

30 application is intended to cover any adaptations or variations of the preferred 
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embodiments discussed herein. Therefore, it is manifestly intended that this 
invention be limited only by the claims and the equivalents thereof. 



P 

if? 



13 



